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ABSTRACT 


A branch and exclude algorithm for solution of the 


N N 

"knapsack problem", max ) v.x, where ) wx, < W and 
i=l i=l 

x. = 0,1 , is presented which requires relatively small 


ah 
amounts of computer running time and core storage allocation. 
In addition, a branch and bound scheme is developed. The 
branch and exclude method is then compared to the branch and 
bound method and to a brancf¥*and:rbet%id method given by 


Kolesar [2]. Computational results are given. 
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digs Introduction 

Stated in precise terms, the knapsack problem is one in 
which one wishes to select from among a finite collection of 
indivisible objects a subcollection which maximizes a linear 
function subject to a linear inequality constrarne. | 2s 
Dantzig [4] gives the following example: a person is planning 
a hike and has decided not to carry more than 70 pounds of 
different items, such as bed roll, geiger counter, cans of 
food, etc. We try to formulate this in mathematical terms. 
Let w. be the weight of the ee object and Vv. be its relative 
value determined by the hiker in comparison with the other 
objects he would like to have on his trip. Let x. = 1 mean 
that the fee item is selected, and x = O mean that it is 


not selected. We express the weight limitation by 


N 


WX; < 70, with x. = O or 1, and we wish to choose the 
i=l 


N 
xX. SO that the total value, } V;xX, 18 a maximum. 
i=l 


The knapsack problem has also been used as a model to 
formulate and solve various aspects of problems arising in 
Capital investment [5], network reliability [6], capital 
buageting [8], and in optimal methods of cutting stock {7]. 
Dantzig also uses this method in [4] to give a delightful 
"proof" that, of all the forms of marriage (monogamy, bigamy, 
polygamy) monogamy is the best of all possible relations. 

The knapsack problem may be formulated as an integer 
programming problem and solved by any one of the following 


techniques: (i) the cutting plane technigue, (ii) techniques 


employing parallel shifts of the objective function hyper- 
plane, (111i) techniques based upon Boolean algebra, and (iv) 
combinatorial methods which enumerate a restricted subset of 
possible integer solutions. For detailed references on these 
methods see Glover [1]. An alternative formulation of the 
problem may be developed through the use of dynamic program- 
ming [9]. The technique to be presented in this paper falls 
into the class of combinatorial methods mentioned in (iv) 
above. 

In combinatorial methods of this type, the computer 
memory and running time requirements are usually quite large 
for problems involving many variables. 

We present here first a branch and bound algorithm that 
is more in the spirit of Land and Doig [3] than is [2]. We 
then develop an algorithm which requires little computer 
storage and running. Computational results are then presented 


for our two algorithms and for Kolesar's method. 
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eS 


Ze Formulation of the Problem 
We formulate the knapsack problem as follows: 


N 
(1) Maximize ) Via 


N 
Subject to ) w.x. < W, 
4, “ivi - 


Where x, = Oy aot Ones 1 ee ee ee N 


We can assume without any loss in generality that the 
constants v, and Ww, are integers. Fractions can be handled 
By multiplying through by a proper factor. Non-posttiveycon— 
stants may be handled by the following method which is due 


GO Glover [1]: 


a 
fete 
Kh 
< 

Iv 


O and w. < 0 
1 


Tale Peihyane a 
Ont E ay 


[A 
[Vv 
oO 


O and w. 
1 


ii) Next reformulate the problem as: 


Maximize 


fr 


Viy 


f2l = 


N 
Subject to A wiy, SW, 


Where i= 0,1 ‘@mwi=1)-2 2. ee N, 


>, Ge ante at san ,W- > 0 
a 1/-i— 
and where Visi 


=x 1 fo “ve wee = 0 
il 1’ -i- 
The following example should illustrate this technique: 


Maximize 3X1 ~ 2X. = 4x. + Xy 


Saisgect to, 2X5 8=—5%- tex xX, < 0 


JEaE 


Using Glover's procedure this problem is reformulated 
as: 
Maximize 3Y4 + 2y 5 = J! 
Subject to 2y¥4 + 3Y5 <4 
and the optimal solution if the integer constraint on Xo abe 


pellaxed ehs10/3 Units with X = (i61/3,0,1). 


EZ 


oe A Branch and Bound Algorithm 
As shown by Dantzig [4] the optimal fractional solution 
EO 


N 
(2) Maximize ) ee 


N 
Subject to ) w.xX, < W 


1S given by 


a = de ike < O 
ab 
Xo). fleet 
7 
W-) we 
i ei peg 
hes 
W 
tc 


rece Yr 1s the least integer (0 < r < N) for which 


) w. > W. EE no r exists we have all 2. — ae 
: i 1 
== LY 
= 0, we have the optimal solution to (1). 
et xX. is fractional the value of the objective function 


is z(l) = ) vv. + vx, + We consider 2(n), the value of 
iL 3G 


the objective function at node n, as the solution to (2) with 
asSigned variables added as constraints. A branch and bound 
scheme is essentially as follows: 

i Label node 1 with z(l). Go to 2. 

2(a). Find the terminal node with the largest value of 
z(n). This is the node at which the next branch- 
ing will take place. Any node (# 1) contains the 
effect of assigning values to variables and solv- 
ing (2) with the assigned values of the variables 
added as constraints. 


HES) 


(ob). If the solution at node n has all integer var- 
iables, we have achieved an optimal solution to 
Pere Stoo. bf not, ogo. to 3. 

3(a). Set n = n+l and some unassigned variable, say Xi, 
equal to 0. Solve (2) with all assigned variables 
added as constraints. Label node n with the value 
Zs BGO ecows(b) , 

(b). Set n = ntl and Ss 1. Solve (2) with all 
assigned variables added as constraints. Label 
node n with the value z(n). Go to 2. 


Kolesar's algorithm consists in taking as x in step 


‘iene 
3(a), the unassigned variable with smallest index (i.e., 

t= i fomewnaeh vi /Ws is the maximum for unassigned variables 
xs). 


We propose instead to follow [3] and to take x in step 


sae 
3(a), the variable that is fractional at node n. 
To illustrate our branch and bound algorithm, we solve 


the problem given in [2]: 


Index W. Re 
al ab 
1 30 60 
2 50 60 
3 40 40 
4 10 10 
5 40 20 
6 30 10 
7 10 3 
and W = 100. Solving (2), we obtain 2(1) = 140 with x, = L3 
Xo = deg X2 = 1/2. We label node 1 with z(1) = 140. Since 


X3 is fractional we branch from node 1 and proceed to step 
3(a). We solve (2) with x, = 0. This produces 2(2) = 135 


S 
with x, = abe Xo = ali, X2 = 0, X, = lise Xe = 1/4. We label node 
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NODE 1 Zt ne 





NCDE 2 NCDE 3 | 2(3)=136 
NODE ¢ NODE 7 NODE k MODE 
NODE 8 NODE 9 


bea] Goa 


Figure 1, Complete tree for the example 


cS 


Pewee z(2) = 135. In step 3(b) we set Xq = 1. This produces 


more> 136 with x. = I> x. = 3/5, x, = 1. We go to step 2 in 


i 2 3 

the algorithm. We see that z(3) is the maximum for all ter- 
minal nodes. Since the solution at node 3 has Xo fractional 
we branch from node 3 and proceed to 3(a). We solve (2) with 
ca 1 and X5 = 0 added as constraints. This produces 


A ele Ore tela Se ee Oe Xx, = 1, x2 = a 


il 2 3 5 
We proceed to 3(b) and solve (2) with > l and x, = l. 
This produces z(5) = 120 with xX, = eZ; X5 = iy X34 = 1. Re- 
turning to step 2 we see that z(2) is the maximum for all ter- 
minal nodes. Since the solution at node (2) has Xe fractional 
we must branch from node 2. The method continues easily and 
the complete tree is shown in Figure l. The optimal solution 
is given at node 8 with z(8) = 133, xX, = Lee X5 = i x, = IL 
Xo = 1 and all other x, = On 

The algorithm achieved solution with the generation of 
9 nodes. The same problem was solved in [2] and required 15 


nodes. Further computational experience is presented in 


Section 5. 


aS 


4, A Branch and Exclude Algorithm 

Branch and bound techniques normally require large 
amounts of computer storage because of the necessity to store 
information for all terminal nodes. In this section, we pre- 
sent a branch and exclude algorithm for solving (1) that re- 
quires little computer storage. 

Our branch and exclude algorithm first finds an obvious 
integer solution to the constraints of (1). This solution 
is a lower bound to the optimal solution. We develop a branch 
of a tree and explore each part of the branch until the lower 
bound is reached or until a new feasible solution is found 
that represents a larger lower bound. We then back-track 
and develop new branches of the tree developing possibly 
larger lower bounds. Further branching is excluded when the 
lower-bound is reached. The algorithm stops when all new 
branches are excluded. The only information that is stored 
is the current lower bound solution and the branch routing. 
At the end, the lower bound solution is optimal. 

Define [x] as the greatest integer less than or equal 
Eo xX. 


Define S(Xp Xp ree eX) as the current lower bound solu- 


N 
tion, where the x. are all given and Xo = ) ye iiees 
) a 34, 21 
Define K(X) Xo 00+ 1 Xy) to indicate assigned variables. 
A value x, = 2 (or any number not equal to zero or one) in- 


dicates the variable is unassigned. An assigned variable 


will have the value zero or one. 


h 


Define R(L) as the index of the iby assigned variable. 


i 


The solution to (2), with L assigned components of X 


added as constraints, is 


x. = ltt ie OR (7) ae (7). Seen... >.) 
a OT Eo meat of oo Rul chime Finn Seder cneny li) 
(v3,) aia =o ta ee ly ecleopendl ng wonmmehc 
assignment 
L 
ee oe “R (i) *R(i) er ne 
Is 
Z(L+1) = a Ve i) *R (i) eee + VX) 
where the set M is given by 
Mi(b)ic= 11 |i. <cepein Zeeks 5) oa) 1) edd 
aiid US ethemlCas Palireg cra Coes Nee lonewinen 
L L 
sacaaiel ee a WR (i) *R(i) 
Pee nO. erleree we have all x, = 1 for i 4 R(j) (9 = 2, ae 


A lower bound to the solution of (1) is given by 


+ } a 


“2 eee) eek ad 


L 
<= ) 
1L= 


i 5 
The algorithm follows: 
1. Set L = 1 and all components of X to two. Go to 2. 


gee colven(2)m Lf the solution is all imteger we have tme 


optimal solution. Stop. If not the solution is xX, = A 

X5 = L,--+,X,_y = 1 and x, is fractional. We calculate 

X) = ) v, and form S(x,,1,1,...,1,0,0,0) as a lower 
neg 


bound to the optimal solution, where the zero components 
in S represent xX, = Creel cee, CCte R(lje— roandathe ae 


Cemponent Or X CO Zero. Go to 3. 


Ws) 





3(a). Solve (2) with the L assigned components of X added 
as constraints. 

We obtaimm (3): Tiael Ze tol X, to to 4. If 
Zi + L)o> Xo and we have an integer solution, take 


x = Z(G + loge torm.a new S(XQ py Xy ree eX jewbrom (3), and 


N 
gomtOu4.:< Li bzautely) sles X, and we do not have an integer 
Seuiitwern;, GO tO el bide 

poe Tf x(L) > Xo take Xo = x(L) and form a new 
S(Xg Xp ree Xy) frome (3)e wrth 2 Ae 0. In any case, Set 
L=L+41, take R(L) = r, and set the R(L) component of X to 
zero. Go to 3(a). 
4(a). If the R(L) component of X is equal to zero change the 
component to one and go to 3(a). If the R(L) component of 
X is one go to 4(b). 

(oe lf L = 1 the optimal solution is S(X_ Xp ree e 1X) 
Stop. If L # 1 change the R(L) component of X to two, set 
L = L-l, and go to 4(a). 

This completes the algorithm, z(L) represents the value 

of the objective function at the ea level of a branch. R(L) 
represents a routing of assignments along the branch. Only 
one branch is studied at a time with preference given to 
asSigning the value of zero to the fractional variable. This 
allows lower-bounds to be achieved more rapidly. The only 
permanent storage information required is the current lower 
bound solution S, the assigned variables R(L) at level L of 


the branch, and the assignment vector xX. 
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In the problem given above the algorithm as viewed in 


Figure 1, goes as follows: 


ie 


aNOr. 


tilly 


laa 


IGS 


14. 


In this 


At node one the optimal solution to (2) is 


fractional. x, = 120, 

Branch to node two, assigning X32 = Ox 

Branch to node six, assigning Xp = OF 

Branch to node eight, assigning Xe = Oreo Phas 
produces an all integer result with Xo = 133% 


Xx, = i X5 = i Xx, = Le Xo = 1 
Back-track to node six; branch to node 9, 


assigning x, = l. 


6 
Back-track to node 6, removing the assign- 
ment on Xe « 

Back-track to node 2; branch to node 7, 
assigning Xp = Ihe. 

Back-track to node 2, removing the assign- 
ment on Xe 

Back-track to node 1; branch to node 3, assign- 
Liga — 13a" Calculate (1) =—-100> do net 
change the current lower bound. 

Branch to node 4, assigning Xo = Ov, 

Back-track to node 3; branch to node 5, assign- 
ing X5 = ae. 

Back-track to node 3, removing the assignment 
on X,. 

Back-track to node 1; end. 


The optimal solution is given at node 8. 


example, the total tree investigated has the same 


number of nodes. 


20 


oye Computational Results 

We have programmed the two branch and bound schemes and 
the branch and exclude algorithm in Fortran IV and have run 
test problems on the IBM 360/67. <A program for the branch 
and exclude method is given in Appendix A. 

The coefficients Vv. and w, were generated as random 
integers and the three methods were then tried on the same 
problem. If 

BK = Kolesar's branch and bound algorithm, 


BB 


the branch and bound algorithm presented here, 
BE = the branch and exclude algorithm presented here, 
we obtained the following results for the number of nodes 


generated and the time to achieve solution: 


TABLE 1. Comparison of Methods 





Wey Nf 20. 730 _™ _40 _™ _50 _fP 20_ _ 30_ _ 40. _ 
BE 28.8 28.0 28.8 BeeT 61 87 120 
251 BB 26.4 20.6 26.6 28.6 4/201 282 395 
_ | BK_ [56.2 _ _49.6 _ _48.0 _ _50.0fj259_ _345_ _447_ 352 
BE 34.5 38.7 36.1 54.5 75 Pel: 145 
50 | BB 38.6 48.2 ADEs) 51.8 11236 361 453 
_ | BK_ [62.2 _ _78.4 _ _78.0 _ _92.6 }{276_ _434_  _533_ 723 
BE 19.8 39.0 27.8 18.5 58 116 Le 
75 1 BB 31.0 B02 D922 25.0 11222 alee 404 
_ | BK_ [62.6 _ 101.2 _ (84.4 _ _61.6 }[280_ _523_ _578_ 616 
BE i? 2 Ae 46.8 29.0 36 97 Ie 
100 | BB 16.6 ° °#£24.8 64.0 50.4 }}191 301 560 
BK_ | 56.6 65.4 136.6 ies): | | Doe 403 ° 806 


Average number of nodes generated. Average time in Milli- 


seconds. (10 problems run to obtain each average) 
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In these test problems superiority of BB and BE over BK 
is clearly apparent with regard to number of nodes generated. 
As for time considerations,the branch and exclude¢ method 
shows to be markedly faster over the other two methods, while 
BB is faster than BK. Various other problems than those 
listed above were run, and in general, the BE method achieved 
solution in about one-half to one-third the time required» by 
the BK and BB methods. 

The branch and exclude method was tried on several prob- 
lems with 5000 variables and solution was achieved in all 
cases in approximately 4 minutes. The other two methods 
failed because of storage limitations. In general, the branch 
and exclude algorithm presented here can rapidly solve ex- 


tremely large problems. 


giz 
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